Automatic Pigment Classification in Painted Works of Art from Diffuse Reflectance Image Data

Information about artists\u27 materials used in paintings, obtained from the analysis of limited micro-samples, has assisted conservators to better define treatment plans, and provided scholars with basic information about the working methods of the artists. Recently, macro-scale imaging systems such as visible-to-near infrared (VNIR) reflectance hyperspectral imaging (HSI) are being used to provide conservators and art historians with a more comprehensive understanding of a given work of art. However, the HSI analysis process has not been streamlined and currently requires significant manual input by experts. Additionally, HSI systems are often too expensive for small to mid-level museums. This research focused on three main objectives: 1) adapt existing algorithms developed for remote sensing applications to automatically create classification and abundance maps to significantly reduce the time to analyze a given artwork, 2) create an end-to-end pigment identification convolutional neural network to produce pigment maps that may be used directly by conservation scientists without further analysis, and 3) propose and model the expected performance of a low-cost fiber optic single point multispectral system that may be added to the scanning tables already part of many museum conservation laboratories. Algorithms developed for both classification and pigment maps were tested on HSI data collected from various illuminated manuscripts. Results demonstrate the potential of both developed processes. Band selection studies indicates that pigment identification from a small number of bands produces similar results to that of the HSI data sets on a selected number of test artifacts. A system level analysis of the proposed system was conducted with a detailed radiometric model. The system trade study confirmed the viability of using either individual spectral filters or a linear variable filter set-up to collect multispectral data for pigment identification of works of art

Estimating top-of-atmosphere thermal infrared radiance using MERRA-2 atmospheric data

Space borne thermal infrared sensors have been extensively used for environmental research as well as cross-calibration of other thermal sensing systems. Thermal infrared data from satellites such as Landsat and Terra/MODIS have limited temporal resolution (with a repeat cycle of 1 to 2 days for Terra/MODIS, and 16 days for Landsat). Thermal instruments with finer temporal resolution on geostationary satellites have limited utility for cross-calibration due to their large view angles. Reanalysis atmospheric data is available on a global spatial grid at three hour intervals making it a potential alternative to existing satellite image data. This research explores using the Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) reanalysis data product to predict top-of-atmosphere (TOA) thermal infrared radiance globally at time scales finer than available satellite data. The MERRA-2 data product provides global atmospheric data every three hours from 1980 to the present. Due to the high temporal resolution of the MERRA-2 data product, opportunities for novel research and applications are presented. While MERRA-2 has been used in renewable energy and hydrological studies, this work seeks to leverage the model to predict TOA thermal radiance. Two approaches have been followed, namely physics-based approach and a supervised learning approach, using Terra/MODIS band 31 thermal infrared data as reference. The first physics-based model uses forward modeling to predict TOA thermal radiance. The second model infers the presence of clouds from the MERRA-2 atmospheric data, before applying an atmospheric radiative transfer model. The last physics-based model parameterized the previous model to minimize computation time. The second approach applied four different supervised learning algorithms to the atmospheric data. The algorithms included a linear least squares regression model, a non-linear support vector regression (SVR) model, a multi-layer perceptron (MLP), and a convolutional neural network (CNN). This research found that the multi-layer perceptron model produced the lowest error rates overall, with an RMSE of 1.22W / m2 sr um when compared to actual Terra/MODIS band 31 image data. This research further aimed to characterize the errors associated with each method so that any potential user will have the best information available should they wish to apply these methods towards their own application

Low-Cost Multispectral System Design for Pigment Analysis in Works of Art

To better understand and preserve works of art, knowledge is needed about the pigments used to create the artwork. Various noninvasive techniques have been used previously to create pigment maps, such as combining X-ray fluorescence and hyperspectral imaging data. Unfortunately, most museums have limited funding for the expense of specialized research equipment, such as hyperspectral reflectance imaging systems. However, many museums have hand-held point X-ray fluorescence systems attached to motorized easels for scanning artwork. To assist museums in acquiring data that can produce similar results to that of HSI systems, while minimizing equipment costs, this study designed and modeled a prototype system to demonstrate the expected performance of a low-cost multispectral system that can be attached to existing motorized easels. We show that multispectral systems with a well-chosen set of spectral bands can often produce classification maps with value on par with hyperspectral systems. This study analyzed the potential for capturing data with a point scanning system through predefined filters. By applying the system and noise modeling parameters to HSI data captured from a 14th-Century illumination, the study reveals that the proposed multispectral imaging system is a viable option for this need

Predicting Top-of-Atmosphere Thermal Radiance Using MERRA-2 Atmospheric Data with Deep Learning

Image data from space-borne thermal infrared (IR) sensors are used for a variety of applications, however they are often limited by their temporal resolution (i.e., repeat coverage). To potentially increase the temporal availability of thermal image data, a study was performed to determine the extent to which thermal image data can be simulated from available atmospheric and surface data. The work conducted here explored the use of Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) developed by The National Aeronautics and Space Administration (NASA) to predict top-of-atmosphere (TOA) thermal IR radiance globally at time scales finer than available satellite data. For this case study, TOA radiance data was derived for band 31 (10.97 μ m) of the Moderate-Resolution Imaging Spectroradiometer (MODIS) sensor. Two approaches have been followed, namely an atmospheric radiative transfer forward modeling approach and a supervised learning approach. The first approach uses forward modeling to predict TOA radiance from the available surface and atmospheric data. The second approach applied four different supervised learning algorithms to the atmospheric data. The algorithms included a linear least squares regression model, a non-linear support vector regression (SVR) model, a multi-layer perceptron (MLP), and a convolutional neural network (CNN). This research found that the multi-layer perceptron model produced the lowest overall error rates with an root mean square error (RMSE) of 1.36 W/m 2 ·sr· μ m when compared to actual Terra/MODIS band 31 image data. These studies found that for radiances above 6 W/m 2 ·sr· μ m, the forward modeling approach could predict TOA radiance to within 12 percent, and the best supervised learning approach can predict TOA to within 11 percent

Towards an Operational, Split Window-Derived Surface Temperature Product for the Thermal Infrared Sensors Onboard Landsat 8 and 9

The split window technique has been used for over thirty years to derive surface temperatures of the Earth with image data collected from spaceborne sensors containing two thermal channels. The latest NASA/USGS Landsat satellites contain the Thermal Infrared Sensor (TIRS) instruments that acquire Earth data in two longwave infrared bands, as opposed to a single band with earlier Landsats. The United States Geological Survey (USGS) will soon begin releasing a surface temperature product for Landsats 4 through 8 based on the single spectral channel methodology. However, progress is being made toward developing and validating a more accurate and less computationally intensive surface temperature product based on the split window method for Landsat 8 and 9 datasets. This work presents the progress made towards developing an operational split window algorithm for TIRS. Specifically, details of how the generalized split window algorithm was tailored for the TIRS sensors are presented, along with geometric considerations that should be addressed to avoid spatial artifacts in the final surface temperature product. Validation studies indicate that the proposed algorithm is accurate to within 2 K when compared to land-based measurements and to within 1 K when compared to water-based measurements, highlighting the improved accuracy that may be achieved over the current single-channel methodology being used to derive surface temperature in the Landsat Collection 2 surface temperature product. Surface temperature products using the split window methodologies described here can be made available upon request for testing purposes